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We investigate solutions to the minimal surface problem with Dirichlet boundary con- 
ditions in the roto-translation group equipped with a subRiemannian metric. By work 
of G. Citti and A. Sarti, such solutions are amodal completions of occluded visual data 
when using a model of the first layer of the visual cortex. Using a characterization of 
smooth minimal surfaces as ruled surfaces, we give a method to compute a minimal 
spanning surface given fixed boundary data presuming such a surface exists. Moreover, 
we describe a number of obstructions to existence and uniqueness but also show that 
under suitable conditions, smooth minimal spanning surfaces with good properties ex- 
ist. Not only does this provide an explicit realization of the disocclusion process for the 
neurobiological model, but it also has application to contructing disocclusion algorithms 
in digital image processing. 
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1. Introduction 

The study of minimal and isoperimetric surfaces in Carnot-Caratheodory spaces has 
recently received a good deal of attention. 2 ' 3: 6_9, 12, 17 ' 18, 20, 21, 24 In the work cited, 
the various authors explore the existence, uniqueness and properties of minimal and 
isoperimetric problems, finding that, at least in specific lower dimensional cases, the 
minimal/isoperimetric surfaces have a rich geometric structure. Moreover, recent 
work of Citti, Manfredini and Sarti 4, 5 has provided a link between the method 
by which the brain completes missing visual data in the first layer of the visual 
cortex (VI) and the solutions to the minimal surface problem in a specific Carnot- 
Caratheodory space, the roto-translation group, that arises in a mathematical model 
of the function of VI. In this paper, we denote the roto-translation group by 1ZT . 
It is homeomorphic to R 2 x S 1 and we will use (x,y,9) as coordinates. Following 
the construction of Citti and Sarti, we define a Carnot-Caratheodory structure on 
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VT by distinguishing a horizontal subbundle, H, given by the span of the following 
two vector fields at each point: 

X^cos^+sin^, X 2 = |. 

These two vector fields bracket generate the tangent bundle and form a distribution 
of contact planes in this three dimensional space. Placing an inner product on H 
which makes {Xi,X 2 } an orthonormal basis for H, we have the standard Carnot- 
Caratheodory distance on VT: 

d cc (a,b)= mi { [ <7',7'>^ 7(0) = o )7 (l) = 6 

where si is the set of absolutely continuous paths whose derivatives, when they 
exist, are in H. 

In this model, a greyscale image, / : ft C K 2 — > M, has a representation in VT 
given by 



{x,y,0) 



= arc; an ( — - - 



If a portion of the image is occluded in a domain f^o C £1, then Citti and Sarti's 
model provides a completion of the occluded region by constructing a minimal 
spanning surface. More precisely, if c C S is the curve in VST associated to <9fi , 
then the completion is given finding the minimal surface in VST that spans c, i.e. a 
minimizer of the perimeter measure. For a C 1 surface, E, given as a level set of a 
C 1 function u, the perimeter is given by 

5»(E) - J V(Xiu) 2 + {X 2 u)HA (1.1) 

Moreover we know that such minimal surfaces satisfy the following partial differen- 
tial equation: 

Xl [ l x * )+x 2 ( , x * u Up. 

\^{X 1 uf + (X 2 ufj \^{X 1 uf + (X 2 ufj 

In addition to this relationship between the minimal surface problem and a 
model for biological image reconstruction, Citti and Sarti provide a reinterpreta- 
tion of a number of existing algorithms for digital inpainting and image comple- 
tion. In particular, Citti and Sarti 5 , and Citti, Manfredini and Sarti 4 , examine the 
variational models of Ambrosio-Masnou 1 , and a variant of the Mumford-Shah func- 
tional and find that, under suitable interpretation in the roto-translation group 
model, minimizing these different functionals is equivalent to minimizing the stan- 
dard Carnot-Carathedory surface area functional given in equation (1.1). In other 
words, finding minimizers of these various functionals is equivalent to solving the 
Carnot-Caratheodory minimal surface problem in the roto-translation group. 
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In light of this unifying theme in the area of vision and image reconstruction, 
we explore the minimal surface problem in a class of groups which include the roto- 
translation group, TZT . Citti and Sarti show the divergence form of the minimal 
surface equation in TZT and we note that the more general framework of Cheng, 
Huang, Malchiodi and Yang 3 shows both the divergence form equation and that 
the smooth minimal surfaces are ruled surfaces. We note that the characterization 
of minimal surfaces as ruled surfaces has generalizations in Carnot groups with two 
dimensional horizontal bundles 12 and Martinet-type spaces 6 . 

In Section 4, we use the basic form of the minimal surface equation to explicitly 
derive the curves the rule smooth minimal surfaces in the roto-translation group. 
Specifically, we show that with respect to the Webster- Tanaka connection, V, asso- 
ciated to a canonical pseudo-hermitian structure on TZT, the surfaces are foliated 
by V-geodesics which, for fixed x ,y ,8 , R and 9^0 take the form: 



Thus, we provide a geometric characterization of smooth minimal surfaces in 
the roto-translation group which in turn yields an explicit paramctrization for every 
such minimal surface. In constrast to the existing methods of constructing minimal 
surfaces, which approximate a minimal surface via a diffusion mechanism, we note 
that this parameterization provides a method for constructing exact solutions to 
the minimal surface problem. 

Second, we turn to understanding the occlusion problem in TZT. As demon- 
strated in the experimental evidence 5 , in the model of VI given by the roto- 
translation group, representations of image data in TZT potentially contain different 
layers of conflicting data due to both modal and amodal completion of the image. 
In light of this finding, we focus on solving the occlusion problem by finding all pos- 
sible smooth solutions or partial solutions of the minimal surface problem with a 
fixed boundary. In Section 6, we develop a test for determining when two points on 
a given curve can be joined by a V-geodesic. We begin with a fixed curve, c C TZT, 
which is the boundary of an occluded region of the representation of an image in TZT 
and is parametrized as c(t) = {(3{t),9{t)). For each point, c(to) on c, we construct 
the set of other points on c accessible to c(i ), denoted £/(c(t ),c). To construct a 
portion of a smooth minimal surface we simply need construct a function: 



where I? is a connected subset of S 1 and so that u(t) <E £/(c(t),c). We note that 
to construct a smooth minimal spanning surface, we must have that D = S . For 
each t, this function give a point c(u(tj) connected to c(t) by a rule. Needless to 
say, there are numerous possibilities that occur when attempting to construct u. In 
particular, we note that we a guaranteed neither existence nor uniqueness of such 



x{t) 

y(t) 
0(t) 



.to + Rsm(6(t)) 
y a + Rcos(6(t)) 

e a + k. 



u : D C S 1 S 1 
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a u. A key tool in the analysis of g/(c(to),c) is the transversality function given by 



To further examine this procedure, we simplify the investigation somewhat and 
restrict our consideration to curve c so that the projection of c to R 2 C R 2 x S 1 = VST 
is a circle. Under this assumption, we are able to describe a number of different cases 
ranging from cases where one can always find such a u, cases with multiple u and 
cases where no such u exists. In each of these cases, we give explicit examples using 
test image data and give some indication as to the cause of the various pathologies. 
The examples provide a number of obstruction to the existence and/or uniqueness 
of smooth minimal spanning surfaces. 

On the positive side, after examining these various cases, we present a theorem 
showing sufficient conditions for when a smooth minimal completion exists. 

Theorem 1.1. Let I : M 2 — > R be an intensity function of an image with an 
occlusion given by a circular region D. Further, suppose 7 £ TZT is the 6 lift of dD 
and that the occlusion is completely nondegenerate and occludes no critical points 
of I. If Q'{t) ^ for t £ [0, 2n] then there exists a minimal spanning surface of 7 
where the projection of each rule of the surface lies in the interior of D. Moreover, if 
Q'(t) < for t £ [0, 2ir] then the projection of this spanning surface to the xy-plane 
is surjective onto the occluded region. 

An occlusion is completely nondegenerate if there are no critical points of I 
on dD, only a finite number of critical points in the interior of D and an angle 
function can be extended continuously across those critical points (see below for a 
more precise definition). 

We again emphasize that, in such a case, the construction of such a surface is 
significantly less computationally intensive than the iterative approximative method 
used by Citti and Sarti 5 . We expect that similar gains can be achieved with respect 
to the other models mentioned above such as the Ambrosio-Masnou and elastica 
methods. In this direction, we note that the authors 11 use a discrete version of the 
method used in this theorem to provide a new algorithm for disocclusion in the 
context of digital image reconstruction. Moreover, we expect that this method will 
have application to neurobiology: by explicitly constructing completions of images, 
we will be able to provide testable hypotheses for neurobiological function of VI. 



Q(t) = 6{t) - ippif) 




= (-sin(^ /3 (t)),cos(^(t))) 



2. Modeling VI via the roto-translation group 

In this section, we review the basic biological findings describing the function of VI 
and describes a mathematical model of VI. Moreover, we describe the connection, 



2, 2008 1:25 



RT-min 



Minimal Surfaces in the Roto-translation group 5 



0!0;0 
CB CB-i <B; CB- 



£> 

€)€)€)€» 


(3 (2) 




T T " , _| 



Hypercolumnofcells 



Fig. 1. A schematic of the structure of VI 



provided by Citti and Sarti 5 , between minimal surfaces in the model space and 
solutions to the problem of amodally completing regions of occluded image data. 

Over the past several decades, the function and operation of the first layer of 
the visual cortex, VI, has become increasingly clear. Early research showed that 
VI contains so-called simple cells that are sensitive to, among other things, bright- 
ness gradients with a particular orientation. These cells are arranged in columns 
sharing the same orientation preference 14 ' 15 and the columns are arranged in hy- 
percolumns which represent all possible orientations. This view was further explored 
and modeled mathematically, 13 ' 23 where the authors modeled the hypercolumnar 
cell structure using a contact manifold. The contact model is based on a simplifying 
assumption that treats each column as a point, ignoring the column structure to 
focus on the hypercolumn structure. Mathematically, they use the manifold R 2 x S 1 
to model the hypercolumn structure by placing a circle of directions above each 
point (x,y) G M 2 . Each point (x,y,6) represents a column of cells associated to an 
(x, y) point of retinal data, all of which are attuned to the orientation give by the 
angle 9. See figure 1 for a schematic of the hypercolumnar structure. 

Early assumptions that cortical connectivity should run mostly vertically along 
the hypercolumns and be severely restricted in horizontal directions, while sup- 
ported by some research, was contradicted by later evidence which showed that 
there is "long range horizontal" connectivity in the cortex. These experiments (see 
for example, Gilbert et al 10 ) indicated that horizontal connections are made be- 
tween cells in different hypercolumns of similar orientation preference. Moreover, 
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experimental evidence showed that there is a stronger preference for communica- 
tion between cells of not only similar orientation preference but for ones that lie 
(roughly) along the axis corresponding to the shared orientation. In other words, 
using the notation of the above model, if (a;, 9) and (y, 9) are points in different 
hypercolumns with the same angle preference 9, communication between the cells 
is preferred if the direction 9 corresponds with the direction of the vector from x 
to y in M 2 . This evidence points towards a geometric structure in this layer where 
communication between adjacent cells is allowable in certain directions, vertically 
and between cells in different hypercolumns of similar orientation sensitivity, and 
vastly restricted in all other directions. This type of situation has been studied in 
a variety of settings including, for example, control theoretic problems where the 
degrees of freedom at a particular point are restricted. 

Petitot and Tondut 22 ' 23 incorporate the these biological findings into their model 
by introducing a contact structure on R 2 x S 1 via the one form uj = dx — 9dy and 
introduce a sub-Riemannian metric associated to the contact two-plane distribution 
to encode the geometry of the model of VI. The plane field given by the kernel of 
u>, span{de,d y + 9d x }, corresponds to the space of allowable directions at each 
point. Notice that these are precisely the vertical direction and the direction which 
links cells in different hypercolumns with the same 9 value. Citti and Sarti 5 use the 
following explicit realization of the roto-translation group, 1ZT: 

• 1ZT is diffcomorphic to R 2 x S 1 with coordinates (x,y,9). 

• The following three vector fields span the tangent space at each point: 

X 1= c s W ^ + sin W | 

(2-1) 

X 3 = -sin W ^ + c 0sW | 

We note that [X 2 ,Xi] = X 3 and [X 2 ,X 3 ] = -X x . 

• For an image / : D c R 2 — > M, its representation, £(/), in 1ZT is given by 
(x, y, 9(x, y)) where (x, y) G D and 9 is given by 

' 1 = (-sin(0),cos(<9)) 



Nil 



We note that this is an explicit realization of the model described above and matches 
with the biological evidence concerning horizontal connectivity. The contact sub- 
bundle in this presentation is simply span{Xi, X 2 }. It is a direct calculation that 
this subbundle gives a contact structure and, placing an inner product on this sub- 
bundle making {Xi,X 2 } orthonormal, we have a standard sub-Riemannian metric 
on 1ZT (see the next section for a precise definition). One of the main contributions 
of Citti and Sarti's adaptation of the cortical model is the use of an explicit lifting 
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Fig. 2. Example of lifting an image to TZT 



function that transforms retinal data into a surface in the cortex and allows direct 
use of the Carnot-Caratheodory structure. 

To explore this further, we investigate the application of the model to an im- 
age. Using the representation above, we see that the direction given by the angle 
(cos(#), sin(#)) points in a direction perpendicular to the gradient of/, i.e. a direc- 
tion tangent to the level sets of /. Thus, if nearby points have the same intensity and 
thereby lie on the same level sets of /, their 9 representations will be the same. As 
9 denotes a position in a hypercolumn of cells over the point (x, y), this echoes the 
biological finding horizontal communication occurs between cells of similar orienta- 
tion specificity and the property that the representation respects level lines reflects 
the biological principle that communication between point (xo, yo, 9) and (x\, y\, 9) 
is permitted if 9 points in the same direction as the vector from (xq, yo) to (x\ 1 y±). 
In figure 2, we give a schematic of lifting a simple image to TZT. Represented are 
two layers of cells of similar orientation preference but different hypercolumns and 
two sections of the image that could plausibly be lifted to those layers. 

Citti, Sarti and Manfredini 4 provide a link between mean curvature flow in Ric- 
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mannian approximates of the sub-Riemannian space and the celebrated Mumford- 
Shah functional. Further, Citti and Sarti 5 provided a link between the flow mech- 
anism and several other models including elastica methods 19 and the Ambrosio- 
Masnou model 1 . Moreover, they showed that surfaces that were asymptotically 
stationary under the flows in the Riemannian approximates to 1ZT are minimal 
surfaces in the sub-Riemannian roto-translation group. As discussed in the intro- 
duction, minimal surfaces in Carnot-Caratheodory spaces have been examined in 
some generality 7 ' 8 ' 12 as well as in more restricted settings. 2 ' 3 ' 9 ' 20,21 

Citti and Sarti 5 use the model of VI to investigate the problem of amodally filling 
in an image when a portion of the image is missing due to occlusion or some other 
factor. To use the model, Citti and Sarti took image data with a portion deleted 
and used an approximation of the flow described above to find a minimal spanning 
surface that "fills" the hole in the image. In other words, they find solutions to 
the minimal surface problem subject to Dirichlet boundary conditions. In the next 
sections, we will examine the minimal surface problem in the setting of a class of 
sub-Riemannian spaces which include the roto-translation group. The main goal 
of this paper is to provide a description of such minimal surfaces and to describe 
method by which they can be constructed in particularly a simple manner. 



3. Notation 

In this section, we fix the basic notation used throughout the paper. Let Q be a 
topologically three dimensional one step graded Lie group. In other words, the Lie 
algebra of left invariant vector fields V splits as 

V = Vo Vi, dimVo = 2, Vi = [V , V ] 
Moreover, we assume the following 

• Q is equipped with a Riemannian metric g, which we sometimes denote in 
inner product notation by (•, •) and which makes the grading orthogonal. 

• V LC is the Levi-Civita connection associated to g 

• Assumption 1: Q is said to satisfy assumption 1 if 

[V ,Vi] C Vo 

• We define a Carnot-Caratheodory distance on Q by 

7(0) = a;, 7(1) = y \ 



< 7 ,7 > 



d cc {x,y) = 7 in ^|y 

where A is the set of all absolutely continuous paths that, where their 
derivatives are defined, have 7' G V. 

We note that if Q is nilpotent, then G is known as a Carnot group. We review two 
special examples. 



Example 3.1. The Heisenberg group. The topologically three dimensional Hcisen- 
berg group, H, is one of the simplest nonabelian nilpotent Lie groups. As a smooth 
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manifold, it is diffcomorphic to R 3 . Using the terminology above, we have the Lie 
algebra given as 

!) = V © Vi 

with Vo = span{Xi, X2} and Vi = span{X^\ where there is a single nontrivial 
bracket operation, [Xi,^] = X3. We note that, as H is nilpotcnt, it is a Carnot 
group. 

Example 3.2. The Roto- Translation group. As the roto-translation group, 1ZT, 
appears centrally in the model of visual processing in VI, we review its abstract 
structure. As a smooth manifold, it is diffcomorphic to M 2 x S 1 and, using the 
terminology above, we have the Lie algebra given as 

() = Vo © Vi 

with Vo = span{Yi, Y 2 } and Vi = span{Y^} where we have the following nontrivial 
bracket operations, [YijYb] = ^3, [^2,^3] = Y 2 . Equation (2.1) in section 2 give a 
particular presentation of 1ZT. We note that 1ZT is not a Carnot group as it is not 
nilpotcnt. 

Given a submanifold S G G, the sub-Ricmannian geometry of S is determined 
by the horizontal normalto S, Nq, which is simply the projection of the Ricmannian 
normal, N, to Vo, the first layer of the grading. Explicitly, if {X\, X 2 , X 3 } is a left 
invariant orthonormal basis of TQ with Vo = span{Xi, X 2 } and S is given as a level 
set ip — then, 

No = proj Vo Af 

= proj Vo {{X W ) X 1 + (X 2 ip)X 2 + (X 3 ip) X 3 ) = {Xnp) X 1 + (X 2 <p) X 2 
We also define the unit horizontal normal: 



<N ,N >^ 

Minimal surfaces in Carnot-Caratheodory spaces have been investigated in a 
number of settings 2, 3 ' 7 ~ 9, 12, 20: 21 . In particular, Daniclli, Garofalo and Nheieu 7 
show that a C 2 hypersurface in a Carnot group G satisfy the following minimal 
surface equation: 

divov = 

where divo is the horizontal divergence operator on V. As shown in the next two 
sections, this equation also characterizes minimal surfaces in the class of groups 
described above which satisfy assumption 1. 
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4. Minimal surfaces in TVT 

Lemma 4.1. If Q satisfies assumption 1, i.e. 

[Vo,Vi]cV (4.1) 

then there exists a global strictly pseudoconvex pseudohermitian structure {rj, J, Q) 
with the following properties 

• H = span Vq is the contact distribution for rj. 

• V = span Vi is spanned by the characteristic vector field T for r\. 

• The Levi metric agrees with g on H and is conformal with constant scaling 
factor to g on V . 

Proof: Let X\ and X 2 be a left invariant orthonormal frame for Vo and set 
T = [Xi,X2] £ V\. By left invariance and the bracket generating property of 
Vo, we see that X\,X 2 ,T form a global orthonormal frame for TQ. Set r\ to be the 
dual 1-form for T with respect to this frame. Then clearly H = Ker rj and strict 
pseudoconvexity is immediate as H bracket generates at 1-step. The additional 
bracket condition (4.1) implies that T^dr\ = 0. Thus the first two properties hold 
automatically regardless of which complex structure J is chosen for H. 

Define J : Vo — > Vo by JX\ = —X 2 and JX 2 = X\. Next we extend J to all of 
TQ by setting JT = and declaring J to be linear over R. The Levi metric defined 

by 

h{X, Y) = dr]{X, JY) + rj(X)r](Y) 
then clearly is compatible with g in the required fashion. 

■ 

In this setting, we can now employ the techniques of pseudohermitian and CR 
geometry. Our key tool is the existence of a canonical connection V, derived indepen- 
dently by Webster 26 and Tanaka 25 , for any strictly pseudoconvex pseudohermitian 
geometry. The defining properties of the connection are as follows: 

• H, T, n, dn and J are all parallel. 

• Tor(X, Y) = drj(X, Y)T for X,Y eH. 

• Tot( JX, T) = JToi{X, T) e H for X e H. 

Computations using this connection are most easily conducted in the moving frame 
approach of Cartan, adapted to this setting by Webster 26 . For this technique, we 
first complexify the contact distribution H and define the space of (1,0) and (0, 1) 
vector fields to be the +i and — i cigenspace of J respectively. The (1,0) vector 
fields are then spanned by 

Z = X 2 -iX x . 

The vector fields Z, Z and T then form an orthonormal (complex) frame for the 
complexified tangent space. The dual frame will be denoted by (, ( and 77. We 
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introduce the connection form uo via the identity 

With respect to our frame the Webster- Tanaka connection and Levi metric can be 
uniquely computed from the following equations 26 : 

• dr\ = ih( A (. 

• dh = ujh + hu). 

• d( = (Auj + T]AT. 

• r = mod (. 

The 1-form r is known as the torsion form. 

This connection proves well adapted to many geometric problems. For the study 
of horizontally minimal surfaces we have the following theorem. 

Theorem 4.1. Suppose S is a non- characteristic surface patch in a Carnot group 
satisfying our structure conditions. The horizontal minimal surface equation for S 

divv = (4.2) 

can be written as 

y Jv Jv = 

where v is the horizontal unit normal. Thus if S satisfies (4.2) then S is ruled by 
horizontal V '-geodesies. 

Proof: The volume form for the Levi metric is given by dV — r\ A dr\. By the 
defining properties of the pscudohcrmitian structure this is a constant multiple of 
the Riemannian volume form. From this we immediately see that dV is parallel 
for V and the divergence operator for dV agrees with the Riemannian divergencve. 
Further it f ollows that T_ii] A dr/ = dr\ and so 

div T = 0. 

Now v, Jv and T form a local orthonormal frame for TQ. A standard formula in 
Riemannian geometry (see for example Kobayashi 16 ) then yields 

div X = tracc(VX + Tor(A, ■)). 

If X is horizontal then the second part of the trace formula vanishes identically 
by the defining properties of the Webster- Tanaka connection. Using our particular 
choice of frame we then see that 

div v = { V u v ,v) + { Wj v v , Jv ) + ( SI tv , v ) 
= -(v, Vj v Jv) 

as the second and third terms vanish because H is parallel and the connection 
is metric respectively. Thus on a non-characteristic, horizontally minimal surface 
patch we have 

VjvJv = (4.3) 



February 2, 2008 1:25 RT-min 



12 Robert K. Hladky and Scott D. Pauls 

everywhere. The integral curves of Jv are therefore V-geodesics. But Jv spans the 
intersection of TS with H. Thus the integral curves of Jv foliate S. 

■ 

Remark 4.2. The divergence form of the mimimal surface equation in VST was 
first shown by Citti and Sarti 5 (see section 2.9 proposition 3.1 of that paper) but 
is also a consequence of the more general psuedoherimitian framework of Cheng, 
Huang, Malchiodi and Yang 3 . We also note that a version of this theorem, showing 
that smooth minimal surfaces arc ruled, was first shown in section 2 of Cheng, 
Huang, Malchiodi and Yang, 3 again in the more general context of psuedoherimetian 
manifolds. We include the proof here for completeness and because it facilitates the 
computations below. 

We shall now apply these techniques to the special case to the roto-translation 
group 1ZT. Here the underlying manifold is K 2 x S 1 and Vo is defined by setting 

Xl = cosd - + sine-, X 2 = - 

and declaring them to be a left-invariant, orthonormal frame for a distribution H. 
The Riemannian structure by defining the transverse vector field, 

T=[X U X 2 ] = sin 9— - cos 6— 
ox ay 

and declaring it to be unit length and orthogonal to X\ and X 2 . The remaining 
commutation relations can then be explicitly computed as 

[X 1 ,T] = 0, [X 2 ,T]=X 1 . 

When we run through the construction of Lemma 4.1 we note that g is exactly the 
Levi metric in this case. The contact form can be explicitly computed as 

i] = sin 9dx — cos 6dy 

and the dual to the complex vector field Z = X 2 — iX\ is 

C = ^ {d6 + i cos 6dx + i sin 9dy) . 
Straightforward computations then yield 

dr\ = cos 8d9 A dx + sin 8d9 A dy = 2i( A C 

% % % — 

d( = -d0 A (- smOdx + cos9dy) = --( A r\ + -i] A (. 

The first identity also follows from the fact that the pseudohermitian structure was 
explicitly constructed to ensure that X\ and X 2 were orthonormal. Since h — 2 we 
can immediately deduce from the 2nd Webster identity that the connection form u> 
is pure imaginary. Thus we can deduce that 

i i- 



i 
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This implies that for the frame X\,X 2 ,T the only non-trivial covariant derivatives 
are in the T direction. By examining the real and imaginary parts of the equation 

v T z = - l -z 

we see Vr^i = \X 2 and VtX 2 = —\X\. 

The horizontal V-geodesics can be computed explicitly. Consider a curve 7 = 
(x,y,0). Thus 

7 = (x, y, 0) = (x cos + y sin 9)X 1 + 9X 2 + (i sin - y cos 0)T. 
Thus if 7 is a purely horizontal curve, we must have 

xsin0 — ycos9 = 0. 

Under this assumption, D t j — if and only if both 9 and x cos 9 + y sin are 
constant. We can then solve the equation 

/sin0 -cos0\ (x\ _ ( \ 
\co$9 sin 9 ) \y) ~ \R ) 

to obtain x — Ro cos 9, y — Rq sin 9. 

• Case 1:0^0. Set R = R /6, then 

x = x c + R sin 
y = y c — R cos 
= O + 9t. 

Here x c = x a — i?sin0 o , y c = yo + i?cos0o. 

• Case 2: = 0. Set R = R , then 

x = x + i?(cos0o)i 
y = y + R(s'm9 )t 
= 0o- 

In the sequel we shall refer to the horizontal V-geodesics (and connected subsets 
of them) as rules. 

5. Missing data and amodal completion 

We next turn to the problem of filling in missing image data. Image data may be 
missing for a number of reasons: one object occludes another, the existence of a 
"blind spot" in the retina or some other physiological failure. In terms of digital 
image processing, data corruption, noise or object occlusion can de facto create a 
domain of missing data. 

Using the roto-translation model for the hypercolumn structure in VI described 
above, image data is lifted to 1ZT and missing data is filled by solving the minimal 
surface problem for the given boundary data. 5 Mathematically, if D C K 2 is an 
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open domain where image data is missing, and c G TZT is the image of dD under 
the lift 0(x, y) defined in section 2, then we wish to find a minimal surface X so that 
<9£ = c. Moreover, as discussed in the previous two sections, a C 2 surface meeting 
these requirements must satisfy the equation: 

divo v = 

where v is the unit horizontal normal to S. Moreover, by theorem Theorem 4.1, 
E must be ruled by horizontal V-geodesics. For the balance of the paper, we will 
consider the following problem: 



Occlusion problem: Given a smooth curve, c, in TZT which is the lift of the 
boundary of an open domain in R 2 , can we find a smooth minimal surface spanning 
c which is ruled by V-geodesics? 



As referenced in the previous sections, this type of problem has been studied 
before in a number of sub-Ricmannian settings. In addition to the observation 
that minimal surfaces in some settings are ruled surfaces, there are a number of 
results further describing the nature of solutions to the minimal surface problem 
with Dirichlet boundary data. Among these results, it is important to note that, 
at least in the Heisenberg 20 and Martinet-type spaces 6 , there are obstructions to 
the existence of smooth minimal spanning surfaces, even if the spanned curve c has 
arbitrarily nice behavior. Moreover the second author demonstrates 21 that solutions 
to the Dirichlet problem for ruled minimal surfaces need not be unique (however, 
Cheng, Huang, Malchiodi and Yang 3 prove a uniqueness result for surface subject to 
certain constraints on the characteristic locus). Thus, as the roto-translation group 
is locally very much like the Heisenberg group, we should expect to see issues with 
both existence and uniqueness. In light of this suggestive evidence, we present a list 
of conditions, each stronger than the next, concerning a smooth minimal spanning 
surface S: 

(I) S exists 

(II) Condition I and any rule connecting two points of c projects to a curve in the 
interior of D 

(III) Condition II and the projection of X to D is surjective. 

(IV) Condition III and X is a graph over D 
(V) Condition IV and X is unique 

We note that if condition II is violated, the rules, upon projection, would present 
potentially conflicting data for points exterior to D while if condition IV is violated, 
there would exist points interior to D with conflicting projected image data. Thus, 
condition III is sufficient to guarantee the existence of a completion of the image 
data (not just a spanning surface in TZT) although there may be conflicting data 
while IV would provide a completion with no conflicting data. However, as, a priori 
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there may be multiple lifts, only condition V would yield a unique completion of 
the image data. 

The evidence cited above and the experimental evidence of multiple simultane- 
ous completions of image data in 1ZT 5 suggests that we should not expect to be 
able to satisfy the more stringent requirements. As we develop the machinery to 
construct such surfaces, however, we will keep each of these conditions in mind. 

6. Ruled surfaces in TZT 

In this section, we begin the investigation of the existence and properties of minimal 
ruled spanning surfaces of curves in 1ZT. As we require the solution to the occlusion 
problem to be a surface ruled by V-geodesics, we first look at the set of points which 
can be connected to a given point by V-geodesics. With this in mind, we make the 
following definition: 

Definition 6.1. For a point p G 1ZT we define the accessible set s/(p) to be the 
collection of points that can be connected to p by a single, horizontal V-geodesic. 

Lemma 6.1. Given a point p = (x ,yo, 9 ), the set of accessible points is given by 
the implicit equation 



Proof: When the connecting V-geodesic is a straight line this is immediate. The 
other case follows easily from the trigonometric identity 



The proof of this identity is an easy exercise with the tangent half-angle formulas. 

■ 

This provides a description of the accessible set of p: 

Lemma 6.2. Every accessible set g/(p) is the image of an embedding of the Mobius 
strip into 1ZT. 

Proof: We shall give two arguments for this result. One purely geometric, the other 
more analytic. 

From Lemma 6.1 we note that each 6*-slice of srf (p) projects to a straight line in 
the (x, y)-plane of gradient tan((6 , + 6'o)/2). As 9 increases this line rotates spanning 
out a helicoid. However since we must identify 9 = 2ir with 9 = and the factor of 
1/2 inside the tan means that there is an orientation switch at the join. Thus si (p) 
is a non-orientable line over S 1 which therefore must be diffcomorphic to a Mobius 
strip. 

A more analytic approach is to consider the V-exponcntial map at p restricted to 
the horizontal distribution. From our explicit description of the horizontal geodesies 





(6.1) 
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passing through p we note that if p — (x ,y , ) then 

exp JaX\ + bX 2 ) =(xo + a/b(sm(0o + b) — sin0 o ) ; 

(6.2) 

y + a/b(cos9 - cos(6> + &)), 6 + b) 

at least when 6^0. When i = 0we instead get 

exp p (aXi) = (xo + acos# , y + asin# , 8 ). (6.3) 

If exp(a, b) = exp(a',6') we must therefore have that b = b' + 2kn (with neither 
being 0) and a'/b 1 = a/b. This later can be summarized as the points (a, b) and 
(a', b') must lie on the same line through the origin with b = b' + 2fc7r. Therefore the 
exponential map is bijective from R x [— w, tt] to g/ (p) provided that the sides of 
the strip are identified via (a, —tt) ~ (— a,7r). This provides an explicit embedding 
of the Mobius strip into 1ZT with image (p) . 

m 

For each point on a curve 7, £^{^{t)) may contain many points of 7 or very few. 
Of most interest to the question of building spanning surfaces are the points j(t) 
where jDg/^ft)) = {j(t)} - i.e. the points that only connect to themselves. These 
points give constraints on the formation of a minimal ruled spanning surface. To 
help understand these points, we make the following definition: 

Definition 6.2. Given an embedded curve 7 and a point p £ 7 we define 

■^(Pj 7) = 7 n (p) , 

the points in 7 accessible to p. A point p such that g/(p 7 j) — {p} is called an 
solitary point of 7. The solitary points of 7 will be denoted ^(7). A point p = j(t) 
such that 7 s H is called a Legendrian point of 7. The Legendrian points of 7 will 
be denoted Jz?(7). We also define the orthogonal points of 7, denoted ^(7) to be 
where 7 G span{Ai, A3}. 

As seen in the definition, there are two types of solitary points, the Legendrian 
points and the non-Legendrian points. We remark that for a Legendrian point p, the 
candidate rule passing through p is tangent to the curve 7 and, as in the Heiscnberg 
group 20 , one can use this as a starting place for building a ruled minimal spanning 
surface, the non-orientability of the accessible sets in the roto-translation case means 
that unlike for the Heisenberg group we cannot deduce that all solitary points are 
Legendrian. Indeed non-Legendrian solitary points present more of a problem as 
the candidate rules will be transverse at such a point. To investigate the structure 
of the set of solitary points further we prove the following lemma. 

Lemma 6.3. For any embedded curve 7, the set ^(7) — Jzf (7) is open. 

Proof: If p G ^(7) — ^(7), then 7 intersects £/ (p) only at p and does trans- 
versely. A small perturbation of the base point p will cause a small perturbation of 
the Mobius strip embedding. As 7 is transverse to £^{p), a small perturbation of 
p cannot increase the number of nearby intersections. Away from p the Euclidean 
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distance of 7 from srf (p) can be uniformly bounded below and so a small perturba- 
tion will not introduce any distant intersections. Thus curve points sufficiently near 
to p will also be solitary. Clearly they will also be non-Legendrian. 

■ 

Lemma 6.4. Suppose 7 is an embedded curve and 7(0) G ^(7) — (7). Then the 
map 



is surjective. 

Proof: This is a topological argument. Suppose 7 is a curve in 1ZT that projects to 
the xy-plane bounding a domain D. Further suppose there exists #0 G S 1 not in the 
image of <d. Then, there exists a neighborhood of 9o, -^o, so that N n Im(Q) = 0. 
By Lemma 6.2, .5/(7(0)) is a Mobius strip. However ^0(7(0)) := £/(j(0))\(M. 2 x N ) 
is orientable. Since 7, by assumption, cannot enter R 2 x Ng we see that 7 lies to one 
side of ^0(7(0)). However, since 7(0) G ^(7) — -^(7), 7 must intersect ^0(7(0)) 
transversely at 7(0). This is a contradiction. 

■ 

For the occlusion problem we shall work exclusively with curves that occur as the 
boundary of a smooth, simply connected bounded region D lifted by the contour 
direction field of an intensity function / : M 2 — > R. The boundary dD is can be 
viewed as the image of an embedding (3 : S 1 — > W 2 . Away from critical points of /, 
we can define the lifting function 9 : S 1 — > S 1 by 



where at each point we choose the branch of arctan which makes 9 continuous. We 
shall that an occlusion is non- degenerate if the number of critical points of / lying 
inside dD is finite and 9 can be extended continuously across each critical point. 
The occlusion is completely non- degenerate if there arc no critical points on the 
boundary. Using this function 9 we construct the curve 7 = ((3,9). In addition we 
define the normal angle function for (3 by 



From the definitions, it is clear that a non-critical point j(t) is Legendrian if and 
only if Q(t) = n/2 + kir, k G Z. Likewise j(t) is orthogonal if and only if Q(t) = kn. 
We record this and another fact in a lemma: 



6 : S 1 -> s 1 

t ~ 9( n (t)) 




= (-sin ^(t), cos^t)). 

From these we construct the transversality function for 7, 

Q(t) = 9(t)- lfip (t). 



Lemma 6.5. For any lift 7 associated to a completely non- degenerate occlusion 
problem, 
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(1) j(t) is Legendrian if and only if Q(t) = -| + kir for some integer k. 

(2) -f(t) is orthogonal if and only if Q(t) — kir for some integer k. 

(3) £(7) is non-empty. 

Proof: For the first item, we note that, by definition, 7 is Legendrian if 7' G H . 
Computing, we have 

( 7 '(t),X 3 ) = (3'(t) ■ (VI) = |/3'(i)|(cos(0(i) -^(i))) = |/3'(*)|cos(Q(i)) 

and so, if we assume ^(t) is Legendrian, we have that, equivalently, [3'(t) ■ VI = 
or Q(t) = 9(t) — (pp(t) = § + kir. The second item, concerning orthogonal points, 
follows in the same way. 

The last item follows immediately from the observation that 

/ VI-dr= / |VI(/3(t))||/?'(t)|cosa = 

JdD JO 

where a is the angle between V/ and /?' . Since the first two terms of the integral 
are strictly positive we must have cos a taking both positive and negative values. In 
particular, this implies there are at least two points in [0, 2tt) where cos(a) = 0. By 
the computation at the outset of the proof, these two points are Legendrian points. 

■ 

Remark 6.3. We note that in the proof, we provide a geometric interpretation of 
Q: it measures the angle between f3' and V/. 

7. Occluded Disks 

For computational reasons, it is useful to restrict attention to curves 7 that are 
lifts of circles in R 2 to the rototranslation group. The lifts we are most interested 
in come from the direction angles of the contours of an intensity plot which has 
an ambiguity associated with the choice of orientation. Given a point (x 1 y) and 
a contour passing through this point at angle 6, it is unclear whether to lift it to 
(x, y, 9) or (x, y,6 + n). Accordingly, for a point p = (x, y, 9) £ TZT we shall define 
its conjugate point to be p = (x, y, 9 + ir) and frequently consider conjugate lifts 7 
and 7 simultaneously. 

Any circular lift 7 can be expressed parametrically in standard form as 

7(i) = (.t + Rcost,y + Rsint,9(t)). (7.1) 

When 7 is understood, we shall frequently refer to a point of p e 7 simply by its 
parameter value with this parametrisation. With this parametrization understood 
we can simplify the transversality function Q to 

Q{t) = 9(t)-t. (7.2) 

Lemma 7.1. For a circular lift 7 = (x,y,9), the non-trivial part of s^(^(t),^) is 
given implicitly by 

Q(t) + Q(u) = (2k + l)ir, fceZ (7.3) 
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and =2/(7(^,7) is given implicitly by 

Q(t) + Q(u) = 2fc7r, fceZ. (7.4) 
Proof: From Lemma 6.1 we see that j(u) G (t, 7) if and only if 

sinu-sini {9(u)+9(t)\ 
= tan . 

cos u — cos t \ 2 J 

Applying the trigonometric identity (6.1) we see that this is equivalent to 

fu + t\ {6(u)+9(t)\ 
cot(^—J=tan(^2_Alj. 

The result then follows easily from standard arguments in trigonometry. A virtually 
identical arguments yields the second part also. 

■ 

Lemma 7.2. Conjugation twist- commutes with the exponential map in the sense 
that. 

expp(a,b) = exp p (—a,b). 
Proof: This follows from direct computation from (6.2) and (6.3). ■ 

Corollary 7.1. IfqE£/(p) then q E g/(p). Furthermore the projections to R 2 of 
the connecting rules match precisely. 

Therefore when connecting points obtained from lifting intensity plots we need 
only consider how points in 7 can be connected to either 7 or 7. We shall write 
t ~ u if either -f(u) of 7(u) lies inside s>/(j(t)). 

Lemma 7.3. The Legendrian points of "f occur precisely where 

Q(t) = tt/2 + kir, keZ. 

The orthogonal points of 7 occur precisely where 

Q(t) = kn, ke 7L. 

Proof: Since 7 is parametrized by (7.1), we can explicitly compute that 

d d 
7 = -Rsm(t)— + Rcos(t)— + 6X 2 
ox ay 

= Rsm{9 - t)X x + Rcos(9 - t)X 3 + 9X 2 - 

For j(t) e ^(7) it is then necessary and sufficient that cos(# — t) = 0. For j(t) € 
^(7) the condition becomes sin(# — t)=0. The result follows easily. 



Corollary 7.2. jSf( 7 ) = jSffy), ^(7) = ^(7). 
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Corollary 7.3. On an implicit plot of all points (t,u) such that t ~ u. Any 
transverse crossing of the leading diagonal u — t occurs at either a Legendrian or 
an orthogonal point. 

Corollary 7.4. If t ~ u and -f(t) 6 -Sf(7) then -f(u) e J^(j)- Furthermore, the 
connecting rule projects to the same circle as 7. 
Ift~u and j(t) 6 ^(7) then j(u) <G ^(7). 

With circular lifts, if two points on 7 are known to be connectable then it is a 
straightforward matter to explicitly describe the connecting rule. If (xo,yo,@o) ~ 
(xi,yi,9i) with #0 =/= 6\ then the connecting rule must have the parametrisation 
(x c + i?sin ip, y c — Rcos ip, ip). Thus we need only solve the matrix equation 



(7.5) 



where the fact that the points are connectable guarantees the existence of a solution. 
Elementary methods yield that generically 

R = r X ° 7. , x c = xq - Rsin9 , y c = yo + R cos 8 . (7.6) 

sin 0i - sin O 

This of course yields two separate connecting rule segments depending on whether 
(p transverses S 1 clockwise or anticlockwise. 

Lemma 7.4. If 7 is the circular boundary of a completely nondegenerate occlusion 
and if R(t) = R(j(t)), then R^O. 

Proof: We may assume, without loss of generality, that (xo,yo,9o) = (0,0,0), 
7(0) = (xq, yo) and 7 is parameterized by arclength. If R is zero, then we must have 
that (#0, yo, #o) is connected to itself. Thus, there exists a finite speed parametriza- 
tion (ci(i), c 2 (t)) so that (ci(0), c 2 (0)) = (0,0) and (7i(ci(i)), 72(02 (t))) is connected 
to j(t) for t close to 0. Using (7.5), if we let R(t) be the radius of the circle connecting 
(7i( c i(^))j 72(c 2 (^))) to 7(i), for a generic choice of t, we must have 

( , 72(c2(*))-72(*) = 7iM*))-7i(*) 

[ ' cos(0(c 2 (t))) - cos(0(t)) sin(0( Cl (t))) - sin(0(t)) 

Assuming that R — > as t — > 0, then we must have that 

Rit) = hm 

*-»o sin(0(c 2 (i)))0(c 2 (i))c 2 (i) - sin(0(i))0(t) 

= lim 7i(ci(t))ci(t)- 7 iW = Q 

cos(fl(ci(t)))(9(ci(t))ci(t) - cos(0(i))0(t) 

As sin, cos, q are bounded and I7I = 1, we have that — > ±00 as t — > 0. However, 
direct computation shows that 

cos(i) 
- sin(i) 
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where is the Hessian of /. If I G C 2 and the occlusion is completely nondegen- 
erate, then 9 must be bounded on 7. Thus, R cannot tend to zero. ■ 
With these initial observations in place, we turn to the task of understanding 
when minimal spanning surfaces exist. We will investigate such surfaces using the 
following blueprint (if possible): 

(1) Construct the function Q as given in (7.2). 

(2) Find all solitary Legendrian points using lemma Lemma 7.3. 

(3) Starting from a solitary Legendrian points, construct connections between 
points on 7 to other points on 7 using the implicit equation 

Q{t) + Q(u) = (2fc + l)7r 

and the matrix equation above. 

Subject to goal of satisfying conditions II and III from section 5. 

As we shall see, any one of these steps and/or goals may be violated. One issue 
we address first is that it may not always be possible to construct such a surface 
by connecting points of 7 to other points of 7. In this discussed above, it is 

natural to instead connect some (or all) points of 7 to points of 7. 

8. The index of Q 

To construct a minimal spanning surface for 7 we choose the a Legendrian point as 
a starting place and attempt to build a monotone function u(t) such that u ~ t by 
following the branch of the implicit plot until we reach the other Legendrian point. 
If we implicitly differentiate either (7.3) or (7.4) with respect to t, we see that 

_ 9'{t) 1 _ Q'(t) 
[) l-9'(u) Q'(uY 
In particular, these implicit plots fail to be graphs over the i-axis precisely when 
either Q'(t) = or Q'{u) = 0. The presence of such points is a necessary condition 
for obstructions to the existence of a monotone function u(t). If we have intervals 
of positive measure where Q'(t) is strictly positive and others where it is strictly 
negative, then we note that a spanning surface will not be a graph over D as the 
change in sign forces the spanning surface to backtrack locally, violating our goal 
IV. Generically, even if such points occur we can still follow a branch of the implicit 
plot and constuct connecting rules, however there will be non-uniqueness issues. 
This approach will only completely fail if the plot fails to be an embedded curve, 
which occurs only at points (t,u) where Q'(t) = = Q'{u). 

If we can construct a monotone function u(t), then the collection of connecting 
rules will form a minimal spanning surface for the lift 7. In practice, there are 
several types of obstructions to this method. 

Of primary importance is the degree of the map Q : S 1 — > S 1 . Since connections 
are made by implicitly solving the equation 

Q(t) + Q{u)=n (mod27r) (8.1) 
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the size of the set t) is intimately related to deg Q. For example if deg Q^O 
then there cannot be any non-Legendrian solitary points as non-zero degree implies 
Q is surjective. Likewise, it is clear that 

|£( 7 )| >2|degQ|. 

Of course, critical points of Q add a seperate pathology which influences the number 
of branches of the implicitly defined function u of (8.1). 

For the occlusion problem, the degree of Q is directly related to the critical 
point theory of the intensity function I(x,y). If the occlusion is completely non- 
degenerate, then there is a well-defined map 

VI 

I fl = j^jpflDS^-S 1 . 

The continuous lift is then given by Ig + ir/2. Hence 

deg Q = deg Ig - 1. 

To explore this further, we shall suppose that I(x, y) has at most one (nearby) 
critical point p which if it exists is contained in the interior of the occluded region 
D. In this instance it follows from the definitions that 

deg Q — indexpW — 1 

if p exists. If there is no critical point in the interior then Ig extends to a continuous 
function on the interior disc. Standard results in algebraic topology then imply that 
deg Ig = and so deg Q = — 1. 

Next, we examine various possibilities for the occlusion problem. We will deal 
primarily with completely nondegencrate curves. 



8.1. Case 1: deg Q = -1, Q' ^ 

As a basic example, we consider the following example: set the intensity function 
as I(x,y) = (1 + x 2 (y — x) 2 )^ 1 whose contour plot looks like an angled cross, 
see Figure 3. The region to be occluded is (x — 2.4) 2 + (y — 2.6) 2 < 1, which is 
over just one branch of the cross. While formally I has critical points everywhere 
along the lines y — x and x = 0, we can replace / with x(y — x) without altering 
the underlying contour plot. With this simplification, we have a completely non- 
degenerate occulsion with no occluded critical points. Thus deg Q = — 1 (and hence 
surjective onto S 1 ). The implicit plot of (8.1) is shown in Figure 4. In this instance 
we see that Q is one-to-one from S 1 to S 1 and so there is only one branch of (8.1), 
which spans the entire range. Moreover, direct calculation shows that Q'(t) ^ for 
all t € [0, 2ir}. The boundary lift 7 has exactly 2 Legendrian points, represented in 
Figure 4 (b) by the intersection of the curve Q(t) with the blue lines Q — n/2 and 
Q = 3n/2. The intersections with the lines Q = 0,2w and Q = ir correspond to the 
orthogonal points. 



1 



1 



2, 2008 1:25 



RT-min 



Minimal Surfaces in the Roto- translation group 23 




Fig. 3. Intensity plot for / = (1 + x 2 (x - J/) 2 ) -1 . 



Since Legendrian points can only connect to other Legendrian points (by corol- 
lary Corollary 7.4), we see from the implicit plot of (8.1) that the Legendrian points 
for 7 are solitary and correspond to the intersections with the leading diagonal in 
Figure 4 (a). Thus we can pick either and construct connecting rules by tracing 
the sole branch of the implicit plot until we reach the other Legendrian point. By 
symmetry every point on the curve has now been connected to another and we can 
build a surface ruled by the V-geodesic segments that project into the interior of 
the occluded region. The projections of these segments provide a contour comple- 
tion through the occluded region. In Figure 4 (c) and (d) we show the full contour 
completion together with the associated minimal lift in the roto-translation group. 
We note that it is possible to show that the surface constructed is a graph over the 
occluded region and hence satisfies condition IV. 

8.2. Case 2: deg Q — —1, Q' has zeros 

The last example worked so well because not only was deg Q — — 1 , but also because 
Q was injective. This was because the directions of the underlying contours were 
relatively uniform. If we move the occluded region in closer to the centre of the 
cross, we lose this uniformity and we find that Q depsite having degree —1 is no 
longer injective. Examining Figure 5 confirms that there are points where Q' = 
(this can also be confirmed by direct calculation) . This is represented by the failure 
of the implicit plot of (8.1) to be a graph over either u or t. When we follow the 
program laid out earlier for constructing minimal spanning surfaces, we find that 
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(c) Contour Completion (d) Minimal Spanning Surface 

Fig. 4. I = (l + x 2 (x - y) 2 )" 1 , Centre=(2.4, 2.6), Radius=l, Connecting 7 to 7. 

some points have multiple connections. The surface then connects to some parts of 
the curve as a ridge. Note this surface satisfies condition II but not III. 

8.3. Case 3: deg Q — 

If deg Q = then there are possible obstructions to even local existence of spanning 
surfaces for 7. This phenomenon occurs due to the presence of non-Legendrian 
solitary points. Non-Legendrian solitary points do not occur in isolation but as 
open sets by Lemma 6.3. Since Q is continuous, the presence of non-Legendrian 
solitary points implies that the image of Q is contained in a narrow (width < ir) 
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(c) Contour Completion (d) Minimal Spanning Surface 

Fig. 5. I = (1 + x 2 (x — J/) 2 ) -1 , Centre=(1.2, 1,4), Radius=l, Connecting 7 to 7. 

band. In particular, this implies that the condition deg Q = is necessary. In the 
situation of a single critcal point of I being occluded, the index of VI must be 1. 
In other words, we must be occluding a local maximum or minimum. 
A simple example is to consider the circular lift 

7(f) = (cost, sin t,t). (8.2) 

Here Q{t) — everywhere and so every point is orthogonal with outward pointing 
orientation. The set of (t, u) that satisfy (8.1) is therefore empty. Every point is 
therefore solitary and non-Legendrian and there is no non-characteristic minimal 
surface that spans even a part of 7. 
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(c) Contour Comple- (d) Close Up (e) Over View 

tion 

Fig. 6. I = (1 + x 2 + 0.9s/ 2 )" 1 , Centre=(0.1, -0.3), Radius=l, Connecting 7 to 7. 

It is clear that the lift (8.2) cannot occur from an occlusion problem as it would 
require the vector field V/ to be rotational and hence non-conservative. However 
gaps in the implicit plot are characteristic of occluded maxima and minima, at 
least in the absence of symmetry. See Figure 6 for an explicit example where / = 
(l + x 2 + 0.9j/ 2 ) -1 and the circle occludes a local maximum of the function. In figure 
6 (a), we see that there are two gaps where there are no connections between 7(f) 
and any other point on the curve. The nature of the gap as part of the minimal 
spanning surface is shown in the remaining graphs. 

8.4. Case 4: \deg Q\ > 1 

When Q has large degree, the phenomena of overlapping contours and immersed, 
discontinuous spanning surfaces occurs naturally even when we are considering only 
rules connecting 7 to itself. In Figure 7 we return to the intensity function I = 
(1 + x 2 (x — y) 2 ) -1 , but with the occluded region shifted to have centre (0.1, —0.3) 
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and radius 1. Since we are now occluding the saddle point (of x(y — x)) at (0,0), 
the degree of Q is —2. 

In this instance we see that Q is now a monotone two-to-one function from S 1 to 
S 1 . This corresponds to there now being two branches of the implicit plot of (8.1) 
in Figure 7 (a) . For convenience of reference we shall refer to the highlighted branch 
as branch I and the other as branch II. In both contour completions there is overlap 
as Legendrian points are crossed while transversing the branches. 
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8.5. Connecting 7 to 7 

For the problem of visual completion we must consider also rules connecting the lift 

7 to its conjugate lift 7. To illustrate this we return to our original example with 
the occluded region being the unit disc centred at (2.4,2.6). We follow the same 
basic program, but instead focus on the orthogonal points as our start and finish 
locations. Unfortunately, this introduces a pathology into the construction of our 
contour completion and minimal spanning surfaces. To progress from one orthogonal 
point to the other along a branch of the implicit plot of (7.4) it is necessary to pass 
through a Legendrian point. The effect of this is to switch which segment of the 
connecting rules projects to the interior of the occluded region. As is seen in Figure 

8 (b) this causes overlaps of the contour completion. If we look at the minimal 
surface formed from these internal rule segments, in order to completely span 7 we 
must continue further along the branch until we return to the original orthogonal 
point. This produces the discontinuous self-intersecting surface of Figure 8 (c) . 
From the perspective of minimal spanning surfaces, it is more natural to allow rule 
segments that project outside the occluded region. As is shown in Figure 8 (d), this 
yields a smooth immersed surface between 7 and 7, but it still self-intersects. 

9. A sufficient condition for effective disocclusion 

The examples of the last section point to several general features of the solutions 
to the occlusion problem. We will now show that under some assumptions on the 
image function /, we can guarantee the existence of a solution to the occlusion 
problem, ie. a minimal spanning surface satisfying condition III. To do so, we need 
some preliminary lemmata. 

Lemma 9.1. Let I : R 2 — > R be an intensity function of an image with a completely 
nondegenerate occlusion given by a circular region D. Suppose there exists a minimal 
spanning surface £ of the occluded region associated to a monotone function t — > 
u{t) and so that the projection of £ to D is not surjective. Then, one of the rules 
of the minimal spanning surface is a circle that, when projected to R 2 lies entirely 
inside D and is tangent to the boundary of D at a Legendrian point. 

Proof: We begin with some simple geometric observations. First, if £ is composed 
of circles of infinite radius (i.e. straight lines), the projection is trivially surjective. 
So, we may assume there are some circles in E that have finitie radii. Second, let j(t) 
be the circle that bounds the occluded region (as above), oriented counterclockwise, 
and let c be a circle of finite radius connecting two points on 7 that is the projection 
of a rule of E. If n is the inward pointing normal to c, then let s(t) — cos(a(t)) where 
a(t) is the angle (in R 2 ) between 7 and n at 7(f). Then, if c connects 7(ti) to 7^2), 
sign(s(ti)) = — sign(s(i 2 )) (sec figure 9 (a)). Moreover, in the degenerate case where 
t\ = £2, the circle c is tangent to 7 (i.e. "f{ti) is Legendrian) and thus s(t,) = 0. 
Since u is monotone (and in particular, one to one) and R, x c , y c are continuous in 
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3 35 4 



(a) t ~ u 



(b) Contour Completion 





(c) Minimal Spanning Surface I 



(d) Minimal Spanning Surface II 



Fig. 8. I = (1 + x 2 (x — y) ) , Centre=(2.4, 2.6), Radius=l, Connecting 7 to 7. 



t, s is well defined and, for < \R\ < 00, s is continuous. At points where \R\ = 00, 
s may have a jump discontinuity as s will switch sign at such points. 

By Lemma 6.5, we have that there are at least two Legendrian points and so, 
by the previous discussion, the function s potenitally switches sign as s is zero 
at a Legendrian point. Consider now a circle passing through such a Legendrian 
point. Then (see figure 9 (b)) if the circle is to lie outside the occluded region, then 
moving in the direction of the parametrization of 7, we must have that s moves 
from positive values to negative values. For s to change sign, we must have that the 
inward normal to the rule c changes direction. As we have seen, this can happen at 
a Legendrian point, but it may also happen if \R\ tends to either zero or infinity. 
By Lemma 7.4, R cannot be zero so we must have that cither we encounter another 
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Legendrian point of this type or \R\ — > oo. Without loss of generality, we may pick 
the two Legendrian points so that one of the two arcs of 7 that they bound contains 
no other Legendrian points of this type. Denote this region by tI (ti ,t 2 ) • Hence, we 
must have that \R\ — > 00 for some to G {t\,t?>). 

To finish the proof, suppose there exists a point x in the occluded region so that 
the projection of £ misses x. For each t, there exists a circle, c t (s), connecting 7(f) 
to x with Ct(0) = j(t) and c£(0) = j^jy (7(t))- Let i?(t) be the radius of this circle 
and 




R(t) if 7 (t) -5t(0) > 
-5(t) if7(t)-5t(0)<0 



So, for the projection of £ to miss x, we must have that sign(cos(a))i?(t) ^ &(t) 
for all t G (ti,i2)- But, by the discussion above, sign(cos(a))i?(f) changes sign on 
this region and hence R(t) must also tend to 00 and so must tend to 00 at to as well. 
By construction, the rule through 7 (to) is a straight line and hits x, contradicting 
the assumption that the circles at the Legendrian points lie outside D (see figure 9 
(c)). 

Lemma 9.2. IfQ'<0 everywhere then the limiting rules at any Legendrian point 
are external to the occluded disc. 

Proof: Suppose not. Then without loss of generality we may rotate and reflect the 
image data to match Figure 10, where we are assuming that the curves are oriented 
to the counter-clockwise direction. Elementary arguments then show that the angles 
marked are indeed Q(t) — ir/2 and tt/2 — Q(u) and that these must therefore both 
be positive. However at the Legendrian point we must have Q = ir/2. This clearly 
violates the condition that Q' < 0. 

■ 

Remark 9.1. We remark that if Q' is positive at a Legendrian point, then the 
circle tangent to the occlusion boundary may indeed lie inside the occluded circle. 

Theorem 9.2. Let L : M 2 — > R be an intesity function of an image with an 
occlusion given by a circular region D. Further, suppose 7 G VST is the 9 lift of dD 
and that the occlusion is completely nondegenerate and occludes no critical points 
of L. If Q'(t) ^ for t G [0, 27r] then there exists a minimal spanning surface of 
7 satisfying condition II. Moreover, if Q'(t) < for t G [0, 2ir] then the minimal 
surface satifies condition III. 

Proof: As discussed above, if the occlusion is completely nondegenerate and oc- 
cludes no critical points then deg Q = — 1. Further, since Q' ^ 0, we have that the 
function u(t), implicitly defined by 

Q{t) + Q{u) = (2fc + l)7r 

is monotone. Thus, u : S 1 — > S 1 is one to one and onto and thus, for each t, 
there is a unique point connecting to to t given by u(t). Using the matrix equation 
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(c) \R\ -> oo 



Fig. 9. Behavior of rules in a spanning surfaces 

7.5 determines the rules joining t to u(t) and, by picking the portion of the circle 
defining the rule to be inside of D when projected to M 2 , we also satisfy condition 
II. 

By Lemma 9.1 and Lemma 9.2, we have that under the assumption of Q' < 0, 
the spanning surface satisfies condition III. 



10. Discussion 

The theorem of the previous section shows that if the behavior of / near the occluded 
region is relatively tame, then we can easily construct a minimal surface that spans 
the occluded region. In figures 11 - 15, we demonstrate an implementation of the 
algorithm described in the previous section in cases where the theorem applies. The 
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Fig. 10. Legendre Gap. 



fj Cj 



l.a Image l.b Occlusion l.c Completion 



Fig. 11. Occlusion of a simple linear image 

images were created by assigning color to the value of various functions (as with 
the previous test image). 

In figure 11, we see that the algorithm easily completes a linear image using 
straigt lines. Similarly, in figure 12, we see that when the a complicated portion of a 
region is occluded, it may not necessarily be recovered. In more interesting examples 
that show the power of this new method, figures 13 ,14, and 15 show completion 
using the circles idenitificd in previous section. Figure 13 shows the completion of a 
curve which preserves the curvature of the original image. Figures 14 and 15 show 
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2. a Image 2.b Occlusion 2.c Completion 



Fig. 12. Occlusion of a complicated portion of an image. 



O O O 



3. a Image 3.b Occlusion 3.c Completion 



Fig. 13. Occlusion of a curve 

that the algorithm preserves concavity. We note that this are features that earlier 
diffusion based algorithms often had trouble completing in a reliable manner. 
All of the other cases presented show different types of pathology: 

(1) Case 2 shows the simplest type of failure of a spanning surface to be a graph: 
when Q' has zeros. In this case, we have "backtracking" of rules which causes 
the resulting surface derived from the implicitly defined function u to give an 
immersed rather than embedded surface. 

(2) Case 3 shows an instance where no smooth minimal surface exists due to the 
presence of non-Legendrian solitary points. As pointed out above, this behav- 
ior seems to be characteristic of occluded critical points of /. Moreover, this 
behavior further indicates that restricting to smooth spanning surfaces, while 
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4. a Image 



Fig. 14. Completion preserves concavity, I 



^ ^ 



5. a Image 5.b Occlusion 5.c Completion 



Fig. 15. Occlusion preserves concavity, II 

computationally effective, will not solve any possible minimal surface prob- 
lem with Dirichlct conditions. Case 2 is similar to a result to the case in the 
Heisenberg group 20 but Case 3 shows entirely new behavior stemming from the 
nontrivial topology of the space. 
(3) Case 4 shows that with higher degree there are potentially both nonuniqueness 
issues as well as problems with satisfying condition IV, i.e. forcing the spanning 
surface to be a graph. 

The pathologies outlined above, coupled with the discussion of connection 7 to 
7, lead us to several conclusions. First, the restriction to smooth spanning surfaces, 
while sufficient for many types of problems (such as those of theorem Theorem 9.2) 
is likely insufficient for more complicated areas of an image. Second, the different 
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1 
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types of pathologies suggest that to amodally complete a given occlusion, the "best" 
completion is likely to come from knitting together various pieces of several differ- 
ent solutions (i.e. from different branches of the curve defining u or from pieces 
connecting 7 to 7) . Again, this points towards the neccesity of a more sophisticated 
mechanism. However, we point out that this is consistent with the simulation data 
found by Citti and Sarti 5 showing that several different possible completions are 
present at the same time in 1ZT after using their diffusion method. If this model 
of minimal surface completion is accurate reflecting the completion mechanism in 
VI, this ambiguity stemming from multiple (partial) solutions may be resolved by 
the input and feedback from other layers fo the visual cortex. In particular, we 
note that our algorithm often produces connections between level sets of different 
"heights" thus creating a completion which is not ideal from the point of view of 
matching like intensities within the image. This is consistent with the model of the 
visual cortex present in section 2 as the representation of the image in TZT does 
not carry information about the intensity of the image, but only information about 
the level sets of the image itself. One expects that with additional input such as 
color /intensity information, the best possible completion could be picked out of the 
possibilities. 
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